Modeling for Automated Cloud Application Testing

Developer testing (also called unit testing) can help ensure the development of high-quality cloud applications. However, unit testing normally can’t take into account all possible inputs because the input space is too large or even infinite. So, developers need criteria to decide which test inputs to use and when to stop testing. Effective use of criteria such as structural-code coverage can help reveal faults. Manually writing test cases to achieve high structural coverage requires developers to consider implementation details, making this task laborious and time-consuming. Also, developers sometimes have problems providing specific combinations of test inputs and cloud states to cover specific paths or blocks. To reduce the manual effort, they can employ automated test generation tools that use dynamic symbolic execution (DSE),1 such as Pex (http://research.microsoft. com/projects/pex), a constraint-solving technique based on path exploration. But these tools fail to generate highcovering test inputs because they can’t generate a required cloud state or can’t control the cloud state. Using stub cloud models could help alleviate these issues (see “Related Work” sidebar). With such models, developers can simulate a real cloud environment with a fake stub that provides default or user-defined return values to cloud-related API method calls. However, these values won’t help achieve high structural coverage, and writing such a model requires much manual effort. A parameterized cloud model extends a stub cloud model to enable generation of appropriate return values for covering different paths or blocks. Typically, a simple parameterized model would assume that every return value was possible as long as it could lead to a new path, even when this return value wasn’t feasible in a real cloud environment. Without reflecting the actual cloud environment’s logic or state consistency, such a model could cause false warnings among reported failing tests. To achieve high structural coverage for cloud applications while causing few false warnings, we’ve developed an approach that combines a simulated cloud model and DSE. It uses